Iryna Schlackow and Borislava Mihaylova
on behalf of the SHARP Collaborative Group
useR! 2019
Toulouse, July 11, 2019
Facilitating external use with user-friendly
interfaces: a health policy model case study
Motivation:
what is a health policy model?
A health policy model is a tool to inform policy decisions by
projecting people‘s life courses. Predictions include
disease events
life expectancy
quality of life
healthcare costs
effects of treatments
- positive (disease risk reduction) and negative (adverse effects)
Projections made over long time periods (eg lifetime)
Motivation:
why are health policy models needed?
Healthcare budgets are limited and not all treatments can be
recomended even if effective
Models show whether treatments are good value for money
Health policy models are increasingly used by policy makers
and clinicians
In UK, cost-effectiveness analyses are required by NICE
- Good-value-for-money: £20-30K per extra quality-adjusted life-year
(QALY)
Flexible models can help answer many policy questions
Aim for transparency, reliability, reproducibility and usability
Motivation:
how to facilitate usability?
Transparency Reliability Usability
Release
the code
😀 😀
☹️
useRs only
code mis-use
Publish equations
and
methods
🙂 🙂
☹️
analysts only
Provide user
-friendly
interface
☹️
black box
☹️
😀
NB: user vs useR
Publish equations and
methods and provide
user
-friendly interface
🙂 🙂 😀
SHARP CKD-CVD model:
Shiny interface
Case study: SHARP CKD-CVD model
Background
Chronic kidney disease (CKD) increases cardiovascular (CV) risk
Want to project long-term outcomes in CKD
cardiovascular events, CKD progression, life expectancy, quality of life,
healthcare costs;
enable implementation of treatments to reduce cardiovascular risk
assess long-term effects and cost-effectiveness.
Patient-level data from a trial
baseline characteristics, within-trial events
Risk equations derived from the data
Combined into a Markov model to do lifelong projections
validated internally and externally
SHARP CKD-CVD model:
need for a user-friendly interface
The model to be useful for NICE, other analysts, clinicians...
User-friendly interface accessible from anywhere
No need for knowledge / installation of R
Adaptation to other scenarios/countries
- national mortality rates
- national healthcare costs
Customising parameters in the current setting
- treatment to be assessed
- population characteristics
- duration of treatment / time horizon
- discount rate
SHARP CKD-CVD model:
Shiny interface
http://one-elevenbooks.com/shiny-or-the-truth/
http://dismod.ndph.ox.ac.uk/kidneymodel/app/
Application accessed via a link
The user only sees the front end
All programs/data stored externally
The front end can be modified using CSS
themes, htmlwidgets, and JavaScript actions
- fancy fonts, links, email addresses etc
- error checking on data entry
SHARP CKD-CVD model:
Shiny interface
SHARP CKD-CVD model:
Shiny interface
SHARP CKD-CVD model:
Shiny interface
SHARP CKD-CVD model:
Shiny interface
SHARP CKD-CVD model:
Shiny interface
SHARP CKD-CVD model:
Shiny interface
SHARP CKD-CVD model:
Shiny interface
SHARP CKD-CVD model:
Shiny interface
User-friendly interface:
help with debugging and transparency
User-friendly interface:
help with debugging and transparency
Face validity debugging
- Easier to do on a user-friendly interface (even for the developers!)
Feedback from external users
Running several models against a reference simulation
- Mount Hood diabetes challenge: models predicting long-term outcomes in
diabetes patients
everyone gets the same tasks (eg change in life expectancy after statin initiation)
core assumptions same for everyone
additional assumptions must be documented in a pre-defined template
the results are presented, compared and (usually) published
user-friendly interface enables replication
SHARP CKD-CVD model:
conclusions
SHARP CKD-CVD model is a novel resource for evaluating health
outcomes and cost-effectiveness of interventions in CKD
User-friendly web-based freely available interface aids model use
Together with the published equations / methods helps ensure
reliability of the underlying code and methods transparency
The user can enter with their own parameter values and perform
calculations in different settings
User’s perspective taken into account:
- simple menus, straightforward navigation, pretty looks
- detailed user-guide
- example input/output files, file descriptions and default values
- error checking at data entry could (partially) prevent inappropriate use
- which parameters should be modifiable?
SHARP CKD-CVD model:
challenges and discussion points
Day-to-day support
- Replying to queries, fixing bugs
- R/package updates may break everything!
- Not updating is not an option (according to our IT team)
Is R the best option for such an interface?
- Might Python be faster and/or have better visualisation capabilities?
- C++?
Do the benefits of releasing the code outweigh the risks?
Acknowledgements
Seamus Kent, Richard Haynes, Jonathan Emberson, Will Herrington, Colin
Baigent, Alastair Gray, Jingky Lozano-Kuehne, Martin Craig, Martin
Landray, Kirsty Reith
SHARP participants, study staff and collaborators!
The SHARP study was funded by Merck/Schering- Plough Pharmaceuticals
(North Wales, PA, USA), with additional support from the Australian
National Health Medical Research Council, the British Heart Foundation,
and the UK Medical Research Council
SHARP CKD-CVD model
http://dismod.ndph.ox.ac.uk/kidneymodel/app/
iryna.schlackow@ndph.ox.ac.uk